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@ Method of processing Image data. 

@ One or more intermediate image lines (L1-1) are inter- 
polated by comparing image pixels on a pixel by pixel or 
multiple pixel basis-one line pair at a time. The pixel com- 
parison provides an address (ADDRESS) for a specific 
image pixel pattern (127) held in memory store (125). The 
image pixel pattern addressed (OUTPUT) is taken from 
memory and used to form a part of the intermediate image 
line or lines being interpolated. The process is repeated 
until the entire intermediate image line or lines has been 
constructed. 
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METHOD OF PROCESSING IMAGE DATA 

This invention relates to a method of processing 
image data in pixels including scanning an original image 
to provide a succession of lines of pixels representative 
of the original image. 

Where image data in the form of analog or binary 
signals is to be handled, the generation of the image data, 
storage of the data, and transmission of the image data to 
one or more output stations where the data may be used to 
produce copies of the original represented by the data, 
presents conflicting problems in terms of the cost and com- 
plexity of the apparatus required versus the quality and 
resolution of the copies produced. Where one attempts to 
optimize the quality and resolution of the copies produced, 
the price paid is usually reflected in increased cost and 
complexity of the image data producing and handling appar- 
atus, i.e. in the data generating apparatus required to 
provide image samples or pixels in greater density, in a 
larger memory to store the image data pending use, and in 
increased data bandwidth on transmission of the data from 
one point to another. On the other hand, where one at- 
tempts to reduce and alleviate cost and simplify the data 
generating and handling apparatus by making fewer samples 
of -ihe original, the savings is usually at the expense of 
decreased image resolution and quality. 

The present invention is intended to overcome 
these conflicting problems in the processing of image data 
in pixels, and is characterised by interpolating at least 
one line of image pixels from one or more known, lines of 
image data pixels by carrying out the steps of: buffering 
a first block of image pixels from the known line of image 
pixels to provide an address; addressing a memory storing 
discrete image patterns with the address to obtain the 
image pixel pattern corresponding to the address; using 
the image pattern obtained from the memory to construct a 
first part of the line being interpolated; and, repeating 
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the preceding steps using succeeding blocks of image pixels 
from the known line to construct succeeding parts of the 
line being interpolated until the line is completed* 

The invention offers the advantage that 
5 resolution of the output image is enhanced while image 
storage and transmission bandwidth requirements are 
reduced . 

One way of carrying out the invention will now be 
described, by way of example, with reference to the 
10 drawings, in which: 

Figure 1 is a schematic view showing interpola- 
tion of additional image pixels in a serial pixel stream of 
image data; 

Figure 2 is a schematic view showing interpola- 
15 tion of additional image pixels both within a line pair of 
pixels and of an additional line of pixels between the line 
pair; . 

Figure 3 is a schematic outline of the image 
pixel interpolating system of the present invention; 
20 Figure 4 is a schematic view showing details of 

the linear interpolator and enhancement filter of Figure 3; 

Figure 5 is a schematic view showing details of 
the image line interpolation system of the present inven- 
tion; 

25 _ Figure 6 is a timing chart of the clock pulses 

for the system of Figure 5; 

Figure 7 is a schematic illustrating a represen- 
tative number of the image pixel patterns stored in memory 
and the addresses therefor; 
30 Figure 8 is an isometric view of an exemplary 

raster input scanning mechanism. 

Figure 9 is a schematic view of an alternative 
image line interpolation system; 

Figure 10 is a timing chart for the system shown 
35 in Figure 9; and 

Figure 11 is a representative illustration of 
image pixel patterns for the system shown in Figure 9. 
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Referring to Figures 1 and 2, there is provided a 
schematic illustration showing interpolation of additional 
image pixels 12 between real or original pixels 10 in a 
serial pixel stream 11 (Fig. D and, in Fig. 2, interpola- 
tion of an additional image line between a pair of 
image lines (L x , L 2 ) as performed by the present invention. 
Pixels herein refer to video image data samples or picture 
elements and may comprise either an analog or binary 
representation of the image value at a point. Pixels 10 
may, for example be obtained through line by line scanning 
of an image bearing original 6 by one or more arrays of 
charge coupled devices 7, commonly referred to as CCDs. 
One CCD is a Fairchild CCD 121-1728 pixel 2-phase linear 
array by Fairchild Manufacturing Company. 

One suitable scanning apparatus is shown in Fig- 
ure 8, and comprises plural CCDs 7, together with lenses 8 
supported so that the viewing fields or areas of the CCDs 7 
overlap one another to assure continuity of scan. CCDs 7 
and lenses 8 are mounted on a movable carriage 9 spacedly 
disposed below a transparent platen glass 13 upon which the 
original document 6 to be scanned is placed. A lamp 14 
mounted on the carriage 9 provides illumination with a mir- 
ror combination 16 provided to guide the light rays reflec- 
ted from the original document to the lenses 8 and the CCDs 
7. "A timing control 18 synchronizes actuation of the CCDs 
with movement of the carriage 9 so that as the carriage 
moves past the platen and the original document thereon, a 
succession of line scans of the original document are made 
by the CCDs. 

Further details of the aforesaid scanning mechan- 
ism may be found in US patent application Serial No. 
793,202, filed May 2nd, 1977- other known 

methods of generating image data or pixels may be envis- 
ioned as for example, a TV camera, etc. 
> The individual pixels obtained by the scanning 

apparatus initially comprise analog voltage representa- 
tions of the gray scale of the area viewed by the indivi- 
dual CCD elements. The CCD elements are driven by a suit- 
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able clock 23 {Fig. 3) such that pixels are generated in a 
serial pixel stream. 

For certain purposes, i.e. image storage, trans- 
mission, etc. it is advantageous to provide minimal numbers 
5 of pixels. At the same time, too few pixels can have ad- 
verse effect^n the quality of the image produced. The 
present invention permits minimal input resolution yet pro- 
vides enhanced image output resolution by interpolating ad- 
ditional or extra pixels 12 between original pixels 10 as 
10 seen in Fig. 1. In this embodiment, a one-dimensional in- 
terpolator 15 interpolates pixels 12 based on the image 
value of adjoining pixels 10, it being understood that in- 
terpolation may be either on an analog or digital (i.e. 1 
or 0) image basis. 
15 in the embodiment shown in Figure 2, an addition- 

al line is interpolated by a two-dimensional interpo- 

lator 17 between line pair I* lf L 2 using pixels 10 and 
pixels 12 or in the alternative original pixels 10 only. 

It will be understood that more than one pixel 12 
20 may be interpolated between adjoining pixels 10 of a line. 
Similarly, more than one line may be interpolated between 
adjoining line pairs. Interpolation of an additional line 
, or lines may be made from a single line (i.e. 1^) rather 
than line pair L^, I^. 
25 _ Referring now to the embodiment shown in Figure 

3, the stream of pixels 10, obtained for example from a CCD 
scanning array 7, is inputted to sample and hold circuit 
24, which may comprise a Datel Corp. Sample And Hold Model 
SHMUH circuit* An exemplary input pixel wave form is 
30 illustrated at 25. Sample and hold circuit 24, which is 
driven in synchronization with the pixel stream by pixel 
clock 23, operates to sample the voltage level of each 
pixel 10 and to produce a voltage level signal representing 
the pixel voltage over a preset time interval t following j 
35 which the next pixel is sampled. The output wave form of 
sample and hold circuit 24 for the exemplary signal input 
is shown in 28 and may be described as a H box car" wave. 
The box car signal output of circuit 24 is fed to a linear 
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interpolator 30 where a linear interpolation is made be- 
tween different pixel voltage levels over^ the time xnterval 
t In essence, interpolator 30 generates" a sloping signal 
line bridging between different voltage levels. ^Th.q^. modi- 
fied box car wave output of interpolator 30 is. shown .at 31- 

"... ..'*'-• ^ J. -J c 

From interpolator 30, the interpolated signal is 
fed to an enhancement filter 36. Filter 36 accommodates 
the relatively rapid dropoff in light transfer efficiency 
of the optical elements in the signal generating apparatus, 
i.e. lenses, by enhancing the high frequencies of the si 9r r 
nal. The output wave form of enhancement filter 36, for the 
exemplary signal input, is shown at 37. 

The enhanced signal 37 is fed to one gate of a 
conventional comparator 40 such as Signetic's Inc. Model 
No. NE 5008 comparator. A preselected voltage threshold 
level is applied to the other input gate of comparator 40. 
For signal voltages above the threshold level, the output 
of comparator 40 is a binary 1? for signals below the 
threshold level, the output of comparator 40 is a binary 0. 
The output wave form of comparator 40 for the exemplary 
signal input 25 is seen at 43. 

The square wave output of comparator 40 is fed to 
a conventipnal D type flip-flop 46 such as Texas Instrument 
Model 74-174 Flip Flop. A square wave clock signal 48, the 
frequency of which is a multiple of pixel clock 23 is in- 
putted to flip-flop 46 from a suitable clock 47. In the 
exemplary arrangement shown wherein an interpolated pixel 
12 is to be provided between each original pixel 10, the 
frequency of clock 47 is twice that of pixel clock 23. 
Signal readings are taken off of the rising or leading edge^ 
of clock signal 48 to produce a pixel output stream 49 
incorporating both original pixels 10 and interpolated 
pixels 12. 

Referring to Figure 4, details of linear interpo- 
lator 30 and enhancement filter 36 are thereshown. Inter- 
polator/ filter 30, 36 comprises a conventional lumped par- 
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ameter delay line or circuit 50 such as Model No. 2214-1000 
by Data Delay, Inc. effective to impose a predetermined 
delay on the box ear signal input from sample and hold 
circuit 24 as the signal passes from the input terminal of 
5 circuit 50 to the output terminal thereof. A series of 
voltage taps 54 permit tapping off of the signal after pre- 
set incremental delays. A termination resistor 51 inhibits 
voltage reflections. 

Taps 54 of delay circuit 50 , which are coupled 

10 through resistors 55, 55' to the positive and negative 
terminal of OP amp 60, provide resistor circuits 56, 57 
coupled to the negative terminal of AMP 60 and resistor 
circuit 58 coupled to the positive terminal of AMP 60. OP 
amp 60 may comprise any suitable operational amplifier such 

15 as an R.C.A. Model No. CA-3100 chip. Resistor circuits 56, 
57 each comprise a group of three parallel resistors 55 
while resistor circuit 58 comprises a group of four resis- 
tors 55 f , in the exemplary embodiment illustrated. Leads 
65, 66 which couple resistor circuit 56, 57 and resistor 

20 circuit 58 to the negative and positive terminal of OP amp 
60 have control resistors 68, 69 respectively in series 
therewith. Lead 66 is coupled through resistor 70 to 
ground. Feedback to the negative terminal of OP amp 60 is 
provided by resistor 73. 

25 The signal input to delay circuit 50 progresses 

through circuit 50 over a predetermined interval, the 
length of which is controlled by the design parameters of 
the delay circuit 50. One suitable interpolation/ enhance- 
ment filter has voltage taps 54 at equi-distant points with 

30 a delay interval of 75 nano seconds per voltage tap and 
with resistors 55 of 4.0 K ohms, resistors 55' of 9.75 K 
ohms, resistor 51 of 500 ohms, resistor 68 of 20.5 K ohms, 
resistor 69 of 3.4 K ohms, resistor 70 of 500 K ohms, and 
resistor 73 of 36 K ohms. 

35 In operation of interpolator/filter 30, 36 on a 

change in strength of the signal input from sample and hold 
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circuit 24, as for example, an increase between pixel volt- 
age levels, the increased signal voltage wave front progres- 
sively appears at voltage taps 54 along delay circuit 50. 
With application of the increased voltage signal to resis- 
tors 55 of circuit 56 in succession, the change in input 
signal strength to the negative terminal of OP amp 60 
causes a drop in the output signal of OP amp 60 (shown at 70 
in waveform 37). The output signal of amp 60 then rises as 
the increased voltage signal is applied successively to the 
resistors 55* of circuit 56 with resulting change in the 
input signal to the positive terminal of OP amp 60 (shown 
at 71 in wave form 37). As the increased voltage signal 
progresses through delay circuit 50 to resistor circuit 57, 
the signal input to the negative terminal of OP amp 60 
undergoes further change resulting in a second dip (shown 
at 72 in wave form 37) in the output signal of OP amp 60. 
The signal output of OP amp 60 then stabilizes (shown at 73 
in wave form 37) until the next change in the voltage level 
of the signal input to delay circuit 50. 

It will be understood that a decrease in the 
voltage level of the signal input to delay line 50 produces 

the opposite effect. 

Referring to Figure 5, for line interpolation, 
binary image data which may or may not include interpolated 
pixels 12 as af oredescribed, is fed to line interpolator 
circuit 98 one line pair at a time. 

In the exemplary arrangement . illustrated, a first 
line pair, designated as LI, L2 are employed for illustra- 
tion purposes. It is understood that succeeding line 
pairs, i.e. lines L2, L3; L3, L4, ... Ln-1^ Ln etc. are 
treated in. the same manner. And while image data is des- 
cribed herein as being processed in blocks of four pixels 
at a time, processing of other size image data blocks may 
be readily contemplated. It is further understood that 
where the original image data is. in analog form, the data 
is converted, as by the interpolation . process described 
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-heretofore, to binary level (i*e. n l" or "0 M ) image data 
prior to input to line interpolator circuit 98. 

In the arrangement shown in Figure 5, image data 
is received in serial pixel form on a line by line basis 
5 from a data source, as for example, an image scanning 
apparatus of the type shown in Figure 8, or from memory 
storage, etc., on data input bus 99. Image data in bus 99 
is inputted via line delay buffer 100 and data bus 103, and 
directly via buffer bypass bus 107, to input buffers 106, 

10 108 respectively of interpolator circuit 98. Buffer 100 
serves in effect to impose a one line delay on the image 
data to permit a preceding line (i.e. line LI) to be com- 
pared with the next succeeding line (i.e. line L2) on a 
pixel by pixel basis. During comparison, as line delay 

15 buffer 100 is serially unloaded, the next line (i.e. line 
L2) is simultaneously loaded into buffer 100 for use in 
comparing that line (i.e. line L2) with the next succeeding 
line (i.e. line L3) . 

A suitable clock 114 provides input pixel clock 

20 pulses 0 ir address load control pulses 0 2 , output pixel 
load control pulses 0 3 , and output pixel clock pulses 
The wave form of clock pulses 0, # 2 ' ^3' #4 for the image 
processing example described herein is shown in Figure 6. 

The image data from line delay buffer 100 and in 

25 bypass buffer 107 is clocked into input buffer pair 106, 
108 on a block by block basis by input pixel clock 0^. It 
will be understood that clock signal jj^, or some other 
clock signal in synchronism therewith, serves to clock 
image data from the data source forward in input bus 99 to 

30 load fresh data into line delay buffer 100 and provide data 
in bypass bus 107. 

In the exemplary arrangement shown, the image 
data blocks are four pixels wide. Input buffers 106, 108 
comprise serial in-parallel put buffers, which in the exam- 

35 pie illustrated, are four bits wide each to provide an 
eight bit address to address buffers 122, 124 of pixel pat- 



-9- 



0006351 



tern memory 125. 

On address load control pulse #2' the content of 
input buffers 106, 108 is read through data buses 121, 123 
into address buffers 122, 124 respectively. Address buf- 
5 fers 122, 124, which comprise parallel in-parallel out buf- 
fers, address a specific image bit pattern stored in mem- 
ory 125. 

Memory 125, which may comprise any suitable Ran- 
dom Access Memory (RAM), Read Only Memory (ROM), etc., has 

10 various image pixel patterns 127, representative samples of 
which are illustrated in Figure 7, stored therein. The 
image data in address buffers 122, 124 addresses the speci- 
fic image pixel pattern 127 associated with that address. 
In the example shown, an eight bit address, the sum of the 

15 blocks of image data in input buffers 106, 108, is used. 

On output pixel load control pulse memory 125 
loads f via data buses 129, 131, output buffers 132, 134 
with the specific image pixel pattern addressed which in 
the present example results in buffers 132, 134 each being 

20 loaded with a specific four pixel block of image data. At 
the same time, clock pulse 0^ causes the pixel pattern in 
input buffer 106 to be read into output buffer 118 through 
data bus 121. 

Output buffers 118, 132* 134 comprise parallel 

25 in-serial out buffers. On output pixel clock pulses # 4 , 
the pixel image data in buffers 118, 132, 134 is serially 
unloaded to a suitable output such as a memory (not shown) 
where the image data, comprising both real and interpolated 
pixels , may be held pending use as for example by a suit- 

30 able marking or copying device. 

The various buffers of interpolator circuit 98 
may comprise any suitable commercially available buff ers . 
In the circuit shown, buffers 100, 106, 108 , 118 , 132, 134 
comprise Texas Instruments Model No. 74-195 buffers, and 

35 buffers 122, 124 Texas Instruments Model No. 74-175 buf- 
fers. Memory 125 comprises Fairchild Inc. RAM Model No. 
93-145 chips. 
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r ; In the line interpolation arrangement illustra- 

ted in Figures 5-7 , the pixel image output comprises pixels 
128 of image data from line L^, together with positionally 
equivalent pixels 130 of image data for lines Ll-1 and Ll-2 
taken from memory 125. As described, the image data con- 
tent of lines Ll-1 and Ll-2 is determined by considering 
the image values of the positionally equivalent pixels 128 
in each of lines Ll, L2. It may, however be desirable when 
interpolating image data for one or more additional lines 
to base the interpolated image data for the additional line 
or lines on the image values of one or more pixels outside 
the positionally equivalent pixels, i.e., on the image 
values of one or more pixels before and/or after the pixels 
128 of lines LI, L2. 

Referring particularly to Figures 9-11, there is 
shown an example of line interpolation wherein the predic- 
ted image pixel patterns for the interpolated line or lines 
(in the example shown, lines Ll-1 and Ll-2) is based upon 
the image value of both pixels 239 immediately preceding 
and succeeding pixels 238, together with the image value of 
pixels 238. As seen in Figure 11, the exemplary image 
pixel patterns 227 thereshown each comprise a four pixel 
block 228 consisting of two pixels for each interpolated 
line Ll-1 and Ll-2. The memory address for the image pixel 
patterns 227 comprises an eight pixel address 233 consist- 
ing of a block of four successive pixels from each of ad- 
joining lines Ll, L2, each block of pixels however, being 
made up of additional end pixels 239 and intermediate 
pixels 238. 

The intermediate pixels 238 in the four pixel ad- 
dress block from line Ll also comprises the pixel output 
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portion of line LI. As a result, the pixel output for lines 
LI, Ll-1, and Ll-2 comprises intermediate pixels 238 from 
line LI and the particular pixel pattern 227 drawn from 
memory 225 for interpolated lines Ll-1 and Ll-2. 

In the arrangement shown in Figure 9, image data 
is received in serial pixel form on a line by line basis 
from a data source, as described heretofore, and inputted 
via line delay buffer 200, data bus 203, and bypass bus 
207, to input buffers 206, 208 respectively of the interpo- 
lator circuit. 

A suitable clock 114' provides input pixel clock 

pulses 0^ , address load control pulses 0 2 ', output pixel 
load control pulses 0 3 ', and output pixel clock pulses 0 4 * . 
The wave form of clock pulses 0^ , 0 2 ' ' $2 * $4 for the 
image processing example described herein is shown in Fig- 
ure 10. 

The image data from line delay buffer 200 and in 
bypass buffer 207 is clocked into input buffer pair 206, 
208 on a block by block basis by input pixel clock fi^ , each 
block being two pixels wide. Clock signal 0^ , or some 
other clock signal in synchronism therewith, is also used 
to clock image data from the data source forward into line 
delay buffer 200 and bypass bus 207. 

In the exemplary arrangement shown, input buffers 
206, 208 comprise serial in-parallel out buffers, which in 
the example illustrated, are four bits wide each to provide 
an eight bit address to address buffers 222, 224 of pixel 
pattern memory 225. 

On address load control pulse 0 2 " ' the content of 
input buffers 206, 208 is read through data buses 221, 223 
into address buffers 222, 224 respectively. Address buf- 
fers 222, 224, which comprise parallel in-parallel out buf- 
fers, address a specific image bit pattern 227 stored in 
memory 225, examples of which are shown in Figure 11. 

On output pixel load control pulse 0 3 ' , memory 
225 loads, via data buses 229, 231, output buffers 232, 234 
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with the j specific.: image pixel pattern addressed which in 
the present .example results in buffers 232, 234 each being 
loaded with a specific two pixel block of image data. At 
the same time, clock pulse #3' loads the intermediate pair 
of pixels in input buffer 206 into output buffer 218 
through data bus 226. 

On output pixel clock pulses 0 A • , the pixel image 
data in" buffers 228, 232, 234 is serially unloaded to a 
suitable output as described heretofore. 

On the succeeding clock pulse 0^ , two fresh 
pixels from lines LI, L2 are loaded into input buffer pair 
206, 208 and the foregoing process repeated. 

In the line interpolation process of Figures 9- 
11 ' the content of the interpolated lines Ll-1 and Ll-2 is 
15 determined from the image values of the pixels in lines LI, 
L2. The portion of lines LI, L2 looked at for interpola- 
tion purposes includes the pair of pixels 238 positionally 
corresponding to the interpolated pixels and a single pixel 
239 from before and after the pixels 238. It will be under- 
stood, however, that the number of pixels 239 looked at may 
be greater than one and further, that only the pixel (s) 
before or after the pixels 238 may be looked at. And, while 
the pixels 238 from lines LI, L2 and in image pixel pat- 
terns 227 are illustrated as being two pixels wide, the 
pixel width may range from a low of one pixel to a number 
greater than the two pixels illustrated in Figures 9-11. 

While in the af oredescribed arrangements, a pair 
of intermediate lines (i.e. Ll-1 and Ll-2) are interpolated 
from adjoining pairs, it will be appreciated that the num- 
ber of lines interpolated may be as few as one. It will 
also be understood that the accuracy of the interpolation 
process decreases with increase in the number of intermed- 
iate lines interpolated. And, while interpolation has been 
described herein using a pair of adjoining lines to predict 
one or more intermediate lines, the af oredescribed interpo- 
lation process may instead employ a single known line as 
the basis for predicting one or more adjoining lines. 
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It is understood that the eight pixel address 
illustrated, the composition thereof as four pixels from 
adjacent lines (i.e. lines LI, L2) , and the two and four 
pixel wide two line interpolating image patterns stored in 
memory are exemplary only, and that addresses of various 
pixel length and composition as well as image patterns of 
various pixel length and/or various numbers of interpolated 
lines may be contemplated instead. 

While interpolation of additional pixels 12 may 
be performed at any point in a data system including the 
data output station or terminal, since the image data used 
comprises analog image data, preferably such interpolation 
is performed at the input station. This avoids the expense 
of storing data in analog form. The image data, which 
following interpolation is in binary form, may then be 
stored and/or transmitted. Transmission of the image data 
from one site, i.e. the input station to another site, i.e. 
an output station, may take place in any suitable manner as 
known to those skilled in the facsimile arts for example. 

Line interpolation, which may be performed at any 
point in a data system including the input station, is 
preferably performed at the data output station or terminal 
to reduce data storage needs and transmission bandwidth. 
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CLAIMS: 

1 . A method of processing image data in pixels including 
scanning an original image to provide a succession of lines 
of pixels representative of the original image 
characterised by interpolating at least one line of image 
5 pixels from one or more known lines of image pixels by 
carrying out the steps of: 

a) buffering a first block of image pixels from said 
known line of image pixels to provide an address; 

b) addressing a memory storing discrete image pixel 
10 patterns with said address to obtain the image pixel 

pattern corresponding to said address; 

c) using the image pixel pattern obtained from said 
memory to construct a first part of the line being 
interpolated; and 

15 d) repeating steps, a, b, and c using succeeding 

blocks of image pixels from said known line to 
construct succeeding parts of the line being 
interpolated until said line is completed. 

20 2. The image data processing method according to Claim 1 
characterised by the additional step of buffering image 
pixels from said known line in blocks having a bit length 
equal to the bit length of said address. 

25 3* The image data processing method according to Claim 1 
or Claim 2 characterised by the additional step of storing 
in said memory image pixel patterns for producing at least 
two interpolated lines at once. 

30 4. The image data processing method according to any one 
of Claims 1 to 3 characterised by the additional step of 
buffering image pixels from adjoining ones of said known 
lines to provide said address. 



5. The image data processing method according to any one 
of Claims 1 to 4 characterised by the additional step of 
buffering image pixels from said known line equal to the 
number of pixel positions in said line being interpolated 
plus additional image pixels from said known line, to 
provide an expanded address. 
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